Deep recurrent neural network-based Hadoop framework for COVID prediction with applications to big data in cloud computing

This paper proposes a particle squirrel search optimisation-based deep recurrent neural network (PSSO-based DRNN) to predict the coronavirus epidemic (COVID). Here, the cloud-based Hadoop framework is used to perform the prediction process by involving the mapper and reducer phases. Initially, the technical indicators are extracted from the time series data. Then, the deep belief network (DBN) is employed for feature selection from the technical indicators. After that, the COVID prediction is done by the DRNN classifier trained using the PSSO algorithm. The PSSO is developed by the integration of particle swam optimisation (PSO) and squirrel search algorithm (SSA). The PSSO-based DRNN is compared with existing methods and obtained minimal MSE and RMSE of 0.0523, and 0.2287 by considering affected cases. By considering death cases, the proposed method achieved minimal MSE and RMSE of 0.0010, and 0.0323 and measured minimum MSE of 0.0049 and minimum RMSE of 0.0702 for recovered cases.

Optimizing computational costs of Spark for SARS-CoV-2 sequences comparisons on a commercial cloud

Cloud computing is currently one of the prime choices in the computing infrastructure landscape. In addition to advantages such as the pay-per-use bill model and resource elasticity, there are technical benefits regarding heterogeneity and large-scale configuration. Alongside the classical need for performance, for example, time, space, and energy, there is an interest in the financial cost that might come from budget constraints. Based on scalability considerations and the pricing model of traditional public clouds, a reasonable optimization strategy output could be the most suitable configuration of virtual machines to run a specific workload. From the perspective of runtime and monetary cost optimizations, we provide the adaptation of a Hadoop applications execution cost model extracted from the literature aiming at Spark applications modeled with the MapReduce paradigm. We evaluate our optimizer model executing an improved version of the Diff Sequences Spark application to perform SARS-CoV-2 coronavirus pairwise sequence comparisons using the AWS EC2's virtual machine instances. The experimental results with our model outperformed 80% of the random resource selection scenarios. By only employing spot worker nodes exposed to revocation scenarios rather than on-demand workers, we obtained an average monetary cost reduction of 35.66% with a slight runtime increase of 3.36%. © 2023 John Wiley & Sons, Ltd.

Segmenting with big data analytics and Python: A quantitative exploratory analysis of household savings

According to the national balance sheets of the most advanced economies, despite a recent sharp decline in per capita net wealth, Italian private households present a higher rate among the wealthiest and least indebted in Europe. Recently, the COVID-19 outbreak caused a new leap in households' savings worldwide, particularly in advanced economies and Italy. This study underlines that using advanced analytics tools, household saving behaviour information, and big data analytics may support data-driven decision approaches addressing the management of complex relationships in the financial arena. More specifically, using exploratory and predictive analyses based on big data analytics and machine learning, this study aims to provide extensive customer profiling in the household saving sector in Italy, supporting a data-driven decision-making approach. A profiling of household savings has been defined using the information provided by big data analysis. To proceed in this direction, the hardware and software requirements necessary to perform data processing were considered in the first phase of the study. Data collection was performed according to the so-called extract, transform, load (ETL) process. The contribution of this study lies in the results obtained in terms of data analytics over a dataset that accounts for the purchasing behaviour of almost 20 million postal savers. The clustering algorithm is highly efficient and scales well for large datasets. K-means clustering can be implemented within the MapReduce computational framework. Therefore, the overall procedure proposed here can be easily extended to big data using parallel computing and software implementing MapReduce, such as Hadoop and Spark. © 2023 Elsevier Inc.

Machine learning approach for corona virus disease extrapolation: A case study

Supervised/unsupervised machine learning processes are a prevalent method in the field of Data Mining and Big Data. Corona Virus disease assessment using COVID-19 health data has recently exposed the potential application area for these methods. This study classifies significant propensities in a variety of monitored unsupervised machine learning of K-Means Cluster procedures and their function and use for disease performance assessment. In this, we proposed structural risk minimization means that a number of issues affect the classification efficiency that including changing training data as the characteristics of the input space, the natural environment, and the structure of the classification and the learning process. The three problems mentioned above improve the broad perspective of the trajectory cluster data prediction experimental coronavirus to control linear classification capability and to issue clues to each individual. K-Means Clustering is an effective way to calculate the built-in of coronavirus data. It is to separate unknown variables in the database for the disease detection process using a hyperplane. This virus can reduce the proposed programming model for K-means, map data with the help of hyperplane using a distance-based nearest neighbor classification by classifying subgroups of patient records into inputs. The linear regression and logistic regression for coronavirus data can provide valuation, and tracing the disease credentials is trial.

New efficient Hadoop scheduler: Generalized particle swarm optimization and simulated annealing

Digital growth during the Corona pandemic has generated massive data. The Hadoop in big data has to be more efficient in resource handling and job scheduling. This article proposes the improved job scheduler which is more efficient in fair job scheduling even with the heterogeneous resources. The faster job execution depends upon the localization too. The nearer the slots are, the faster is the execution. So, this article proposes a hybrid metaheuristic algorithm with fair scheduling and data locality as the two objectives in job scheduling. The dominant resource fairness policy in Hadoop YARN is updated by hybrid generalized particle swarm optimization and simulated annealing for minimum locality and maximum fairness in scheduling. The algorithm is tested on various workloads for heterogeneous resources.

Docker Container based Crowd Control Analysis Using Dask Hadoop Framework

Crowd control is a public policy technique in which massive crowds are handled in order to avoid the emergence of possible issues or threats caused by COVID-19 and over-crowding. In this pandemic, social distancing is critical as there is a high chance of being infected in a crowd. With mounting fears about public disease transmission, the significance of crowd monitoring is crucial in these testing times. In the existing system, the model takes more time and resources to process the data from the crowd control application thus resulting in delayed prediction. Early prediction of the crowd level will help people and other government agencies to control and monitor the crowd. Hence, the main goal of the proposed system is to process a large amount of input from the crowd control application in minimal time using Dynamic Task Scheduling (Dask) based Hadoop framework in a multi-node docker cluster. The multi-node cluster processes the input data in different clusters. Each cluster data is fed to model for prediction and forecasting the count of crowd at a location. The models considered for evaluation are RNN_LSTM and ARIMA. The results shown that RNN_LSTM model has provided better accuracy of 97% compared to the ARIMA of 89%. The results show that the prediction performance of RNN_LSTM has shown 40% decrease in Mean Absolute Error (MAE) and 30% decrease in Root Mean Squared Error (RMSE) over the existing ARIMA model. The proposed system is available as an application to the public and enable them to decide whether to visit a particular place or not. © 2022 ACM.

AISAR: Artificial Intelligence-Based Student Assessment and Recommendation System for E-Learning in Big Data

Educational systems have advanced with the use of electronic learning (e-learning), which is a promising solution for long-distance learners. Students who engage in e-learning can access tests and exams online, making education more flexible and accessible. This work reports on the design of an e-learning system that makes recommendations to students to improve their learning. This artificial intelligence-based student assessment and recommendation (AISAR) system consists of score estimation, clustering, performance prediction, and recommendation. In addition, the importance of student authentication is recognised in situations in which students must authenticate themselves prior to using the e-learning system using their identity, password, and personal identification number. Individual scores are determined using a recurrent neural network (RNN) based on student engagement and examination scores. Then, a density-based spatial clustering algorithm (DBSCAN) using Mahalanobis distance clustering is implemented to group students based on their obtained score values. The constructed clusters are validated by estimating purity and entropy. Student performance is predicted using a threshold-based MapReduce (TMR) procedure from the score-based cluster. When predicting student performance, students are classified into two groups: average and poor, with the former being divided into below- and above-average students and the latter into poor and very poor students. This categorisation aims to provide useful recommendations for learning. A recommendation reinforcement learning algorithm, the rule-based state–action–reward–state–action (R-SARSA) algorithm, is incorporated for evaluation. Students were required to work on their subjects according to the provided recommendations. This e-learning recommendation system achieves better performance in terms of true-positives, false-positives, true-negatives, false-negatives, precision, recall, and accuracy.

Dynamic Analysis of Demographic Sentiment

There is no doubt that big data analysis has a very positive impact on economics, security, and other aspects for countries and enterprises alike. Where we have recently noticed the frantic competition between companies to increase their profits by analyzing the largest amount of data as quickly as possible. Especially analyzing data related to Covid-19 to make the most of information in all areas. Covid-19 has drastically affected many lives in recent years but, even in these hard times, businesses can leverage the current pandemic to make a profit. In this paper, we investigate a variety of tweets using MapReduce, Spark, and Machine Learning methods to determine the sentiment of a given tweet based on the information provided by the dataset. With this information, businesses could learn how to present Covid-19 and pandemic related goods and information in a way that will be well received by its audience. To take this a step further, we will investigate trends in sentiment across demographics tweeting about the virus. This information in sentiment is dynamically useful to understand how specific audiences feel about the pandemic. We explore which Machine Learning methods produce the best results such as Multi-Layer Perceptron neural networks and Logistic Regression. © 2022 IEEE.

Arabic Semantic Similarity Approach for Farmers’ Complaints

Semantic similarity is applied for many areas in Natural Language Processing, such as information retrieval, text classification, plagiarism detection, and others. Many researchers used semantic similarity for English texts, but few used for Arabic due to the ambiguity of Arabic concepts in both sense and morphology. Therefore, the first contribution in this paper is developing a semantic similarity approach between Arabic sentences. Nowadays, the world faces a global problem of coronavirus disease. In light of these circumstances and distancing's imposition, it is difficult for farmers to physically communicate with agricultural experts to provide advice and find suitable solutions for their agricultural complaints. In addition, traditional practices still are used by most farmers. Thus, our second contribution is helping the farmers solve their Arabic agricultural complaints using our proposed approach. The Latent Semantic Analysis approach is applied to retrieve the most problem-related semantic to a farmer's complaint and find the related solution for the farmer. Two methods are used in this approach as a weighting schema for data representation are Term Frequency and Term Frequency-Inverse Document Frequency. The proposed model has also classified the big agricultural dataset and the submitted farmer complaint according to the crop type using MapReduce Support Vector Machine to improve the performance of semantic similarity results. The proposed approach's performance with Term Frequency-Inverse Document Frequency-based Latent Semantic Analysis achieved better than its counterparts with an F-measure of 86.7%.

An intelligent distance learning framework: Assessment-driven approach

Online education is popular for various remote trainings, or it becomes inevitable in anomalous situations, for example, COVID-19 pandemic. One of the well-recognized online education platforms is massive online open courses, whose contents are developed by world-famous experts. However, the learning effectiveness of online education is not yet higher than face-to-face classroom education. It is in part because the online contents are uniformly designed for all heterogeneous online students, even to the students who need the same course topic at different levels. Also when a student needs to revisit a course content to strengthen his or her weakness, typical online infrastructures do not pinpoint their areas of weakness to go over. This chapter proposes both online lecture and mobile assessment platforms to elevate the quality of distance learning. The online platform proposed is the three-layers of four quadrant panes. The online platform has three layers: (1) basic, (2) advanced, and (3) application per course module. Each layer is divided into four quadrants: (a) slides quadrant, (b) videos quadrant, (c) summary quadrant, and (d) quizlet quadrant panes. Students begin with the basic layer first, dive into the advanced layer, and apply the application layer. The other platform is a mobile assessment. One of the critical issues in distance learning is fair assessment management. Smartphone-based assessment of online student learning performance disables the high chance of cheating schemes and enables the building of student learning patterns. Its analytics eventually leads to the reorganization of online course module sequences. The contribution is (1) an accurate recognition of student weakness;(2) an intelligent and automatic answering to student questions;and (3) a mobile phone application-based assessment. © 2021 Elsevier Inc. All rights reserved.

Performance Comparison of Apache Hadoop and Apache Spark for COVID-19 data sets

Nowadays the world is fighting against a global pandemic Covid-19 that has resulted in more than 5 million deaths and badly impacted world economy. The global spread of COVID-19 has triggered innovative research in the field of distributed computing using Big Data management tools. Big data analytics tools are used to better understand virus spread, to detect and track Covid-19 symptoms, to estimate risk factors, symptoms, diagnostics and other vital information and to control its spread. This paper presents a review of big data solutions that has been adopted to solve research issues in healthcare by performing distributed computing on massive datasets. In the proposed work, Apache Hadoop with MapReduce framework and Spark is used to perform analytics on Covid-19 datasets in parallel and distributive manner. Both frameworks have configuration parameters which can be modified to facilitate job performance and efficiency. This paper compares the performance of two major Bigdata platforms Hadoop and Spark. The execution time and throughput of both frameworks are analyzed with different input data size. The results shows that both platforms can be used to effectively to process huge amount of data in parallel and distributed computing and the performance depends on size of input data and configuration parameters. The results show that Spark has significantly faster computation time than Hadoop for smaller data sets. © 2022 IEEE

Sentiment Analysis Using Machine Learning and Deep Learning on Covid 19 Vaccine Twitter Data with Hadoop MapReduce

The Coronavirus, also known as COVID-19, initially surfaced in Wuhan, China, in December of 2019. The virus was one of the most widely discussed subjects on social media. As a result, these social media sources are exposed to and present a variety of viewpoints, beliefs, and feelings. Big data is a significant resource for computer scientists and scholars who want to understand how people feel about current events. We present a real-time implementation of a system that can identify Twitter opinions about the COVID-19 Vaccine using Hadoop in this work. All tweets are divided into three categories (Positive, Neutral, and Negative). Sentiment analysis was conducted by Logistic Regression, Random Forest, Deep Neural Network, and Convolutional Neural Network. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

IoT Cloud Computing Middleware for crowd monitoring and evacuation

Map-Reduce is a programming model and an associated implementation for processing and generating large data sets. This model has a single point of failure: the master, who coordinates the work in a cluster. On the contrary, wireless sensor networks (WSNs) are distributed systems that scale and feature large numbers of small, computationally limited, low-power, unreliable nodes. In this article, we provide a top-down approach explaining the architecture, implementation and rationale of a distributed fault-tolerant IoT middleware. Specifically, this middleware consists of multiple mini-computing devices (Raspberry Pi) connected in a WSN which implement the Map-Reduce algorithm. First, we explain the tools used to develop this system. Second, we focus on the Map-Reduce algorithm implemented to overcome common network connectivity issues, as well as to enhance operation availability and reliability. Lastly, we provide benchmarks for our middleware as a crowd tracking application for a preserved building in Greece (i.e., M. Hatzidakis’ residence). The results of this study show that IoT middleware with low-power and low-cost components are viable solutions for medium-sized cloud computing distributed and parallel computing centres. Potential uses of this middleware apply for monitoring buildings and indoor structures, in addition to crowd tracking to prevent the spread of COVID-19. © 2021, North Atlantic University Union NAUN. All rights reserved.